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@ Image movement compensation apparatus. 



@ An image movement compensation apparatus 
(MC) maintaining high image quality is achieved 
using a filed memory (4) having more pixels than the 
number of final output image (Svc) which is deter- 
mined according to the broad cast system or the 
system operation clock. The image movement com- 
pensation apparatus (MS) further has an image pick- 
up device (1) having more pixels than the filed 
memory (4), an image movement detector (5), and 



an image movement predictor (6). The image (la) 
obtained by the image pick-up device (1) is cropped 
according to the image movement predicted by the 
image movement predictor (6). The residual move- 
ment component left in the image (Svb) read from 
the image pick-up device (1) is then detected by the 
Image movement detector (5), and is next removed 
by image movement compensation using the field 
memory (4); 
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BACKGROUND OF THE INVENTION 
1. Field of the Invention 

The present invention relates to an innage 
movement compensation apparatus used to com- 
pensate for hand movement of the Image recording 
apparatus during recording. 

2- Description of the Prior Art 

When subjects are imaged using an image 
recording apparatus that is held by hand or is 
mounted on a moving body such as a, motor ve- 
hicle, movement of the image recording apparatus 
during recording also appears in the recorded im- 
age. Image movement compensation, also known 
as image stabilization, apparatuses that detect this 
Image movement and adjust the recorded image 
position based on the detected movement Informa- 
tion have therefore been developed and are today 
commonly available in consumer products. 

These conventional image movement compen- 
sation apparatuses have been described in "Auto- 
matic Image Stabilizing System by Full-Digital Sig- 
nal Processing" (K, Uomori, et al., IEEE Transac- 
tions on Consumer Electronics. Vol. 36, No. 3. 
August 1990 (pp. 510 - 519)), and in "Electronic 
Image. Stabilizer for Video Camera Use" (T. 
Kinugasa. et al.. IEEE Transactions on Consumer 
Electronics. Vol. 36. No. 3, August 1990 (pp. 520 - 
525)). 

In the "Automatic Image Stabilizing System by 
Full-Digital Signal Processing" described by K. 
Uomori et al.. image movement is detected from 
the video signal of the current field obtained from a 
solid-state image pick-up device, and image move- 
ment caused by hand movement or other cause is 
removed from the current field Image by shifting 
the field memory data read address according to 
the detected image movement. Electronic zoom 
processing is also applied at this time because the 
size of the Image frame changes when reading the 
image signal from the field memory. 

T. Kinugasa, et al., described two different 
methods in "Electronic Image Stabilizer for Video 
Camera Use." In the first. Image movement is 
detected from the video signal of the current field 
obtained from a solid-state image pick-up device, 
and Image movement caused by hand movement 
or other cause is removed by shifting the position 
from which the video signal is read from the solid- 
state Image pick-up device in the next field accord- 
ing to the detected movement. The compensation 
function of this first system, however, does not 
work well with image movement caused by high 
ifrequency hand movement, and the second system 
described therefore provides a buffer memory. A 



motion detection means detects residual hand 
movement in the image read from the solid-state 
image pick-up device to control image reading 
from the buffer memory and thereby correct this 

5 residual movement. 

In the system described by K. Uomori et al., 
however, it is necessary to electronically enlarge 
the image because the movement compensation 
process makes the image frame of the compen- 

10 sated image smaller than that of the original image 
read from the field memory circuit. The problem 
here is that this electronic enlarging process de- 
grades the image quality. 

Furthermore, in the first system described by 

75 T. Kinugasa et al., high precision movement com- 
pensation is not possible because controlling the 
video signal read position of the solid-state image 
pick-up device is based on the detection results 
obtained from the previous field, and movement 

20 compensation is thus delayed by one field. In addi- 
tion, both systems described by T. Kinugasa et al.. 
rely on reading the video signal from either the 
solid-state image pick-up device or buffer memory 
for movement compensation. As a result, the best 

25 compensation precision is limited to one pixel hori- 
zontally or one line vertically even if the motion 
detection precision Is better than one pixel horizon- 
tally or one line vertically, and more precise com- 
pensation is not possible. 

30 

SUMMARY OF THE INVENTION 

The object of the present invention is therefore 
to provide a image movement compensation ap- 
35 paratus which solves these problems. - 

The present invention has been developed with 
a view to substantially solving the above described 
disadvantages and has for its essential object to 
provide an improved Image movent compensation 
40 apparatus . 

In order to achieve the aforementioned objec- 
tive, an image movement compensation apparatus 
for compensating the movement of an image com- 
prises an Image producing means for producing a 
45 first image representing an object expressed by a 
first predetermined number of pixels and reading 
out a second image expressed by a second pre- 
determined number smaller than said first predeter- 
mined number of pixels therefrom; a memory 
60 means for storing said second image and reading 
out a third Image expressed by ,a third predeter- 
mined number smaller than said secorid predeter- 
mined number of pixels therefrom; a movement 
signal means for producing a vector signal repre- 
ss senting the movement of said image at intervals of 
predetermined period; a prediction signal means 
for producing first and second prediction signals 
predicting the movements which said first and sec- 
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ond images which will make in said predetermined 
period, respectively, based on said vector signal; 
and a designation means for designating areas of 
said memory means and said Image producing 
means from which said third and second images 
are read out based on said first and second perdi- 
tion signals, respectively. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other objects and features of the 
present inventipn will become clear from the follow- 
ing description taken in conjunction with the pre- 
ferred embodiments thereof with reference to the 
accompanying drawings throughout which like 
parts are designated by like reference numerals, 
and in which: 

Fig. 1 is a block diagram showing an image 
movement compensation apparatus according to 
a first embodiment of the present invention; 
Fig. '2 is a block diagram showing an image 
movement predictor of the image movement 
compensation apparatus of Fig. 1; 
Fig. 3 is a block diagram showing a modification 
of the image movement predictor of Fig. 2; 
Fig. 4 is a block diagram showing a detailed 
construction of the image movement predictor of 
Fig. 2; 

Fig. 5 is a block diagram showing a modification 
of the movement predictor of Fig. 4; 
Fig. 6 is a block diagram showing an image 
interpolator of the image movement compensa- 
tion apparatus of Fig. 1; 

Figs. 7A, 7B, 7C, and 7D are illustrations of 
assistance in explaining the sizes of images 
obtained in am image pick-up device, a field 
memory, and the final output image according to 
the present invention; 

Fig. 8 is an illustration of assistance in explain- 
ing the image movement compensation opera- 
tion according to the present invention; 
Figs. 9A, 9B, and 9C are graphs of assistance in 
explaining the operation of the movement pre- 
dictor shown in Fig. 4; 

Figs. 10A, 10B, and IOC are graphs of assis- 
tance in explaining the operation of the move- 
ment predictor shown in Fig. 5; 
Fig. 11 is an illustration of assistance in explain- 
ing the operation of the image interpolator of 
Fig. 6; 

Fig. 12 is a block diagram showing an image 
movement compensation apparatus according to 
a second embodiment of the present invention; 
Fig. 13 is a block diagram showing an image 
movement predictor shown in Fig. 12; and 
Fig. 14 is a block diagram showing a attenuation 
compensator of the image movement compen- 
sation apparatus of Fig. 12. 



DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

Referring to Fig. 1, an image movement com- 
5 pensation apparatus according to a first embodi- 
ment of the present invention is shown. The image 
movement compensation apparatus MC Includes 
an image pick-up device 1 for converting optical 
images received through a lens system to elec- 
70 tronic images, i.e., a video signal Svb. The image 
pick-up device 1 has more pixels than a conven- 
tional image pick-up device of which the number of 
pixels is determined according to the broadcast 
system or the system operating clock. The image 
15 pick-up device 1 can be made by, for example, a 
solid state element such as a CCD, but not limited 
to it. 

An analog signal processor 2 is provided for 
applying analog signal processing to the video sig- 

20 nal Svb from the image pick-up device 1. Thus 
analog processed signal is converted to a digital 
video signal Svb by the A/D converter 3, and is 
then transferred to a tield memory 4 and an image 
movement detector 5. 

25 The field memory 4 has a storage capacity 
less than the number of pixels in the image pick-up 
device 1 and greater than the number of pixels 
required for the final video output from an image 
movement compensation apparatus MC. The field 

30 memory 4 temporarily stores the digitized video 
signal Svb. 

The image movement detector 5 detects the 
movement in the image between consecutive im- 
age fields from the digitized video signal Svb and 

35 produces a detected difference vector signal Sm 
representing the image movement. The detected 
difference vector signal Sm is transfen'ed to an 
image movement predictor 6 and a difference vec- 
tor integrator 7. 

40 Based on the detected difference vector signal 
Sm, the image movement predictor 6 produces a 
predicted movement signal Sp which predicts the 
image in the next filed. An image pick-up device 
controller 8 is provided for producing a first reading 

45 address signal Sb based on the predicted move- 
ment signal Sp. The reading address signal Sb 
defines the address for reading the image signals 
from the image pick-up device 1 to out put the 
video signal Svb having the number of pixels that 

50 can be stored by the field memory 4. 

The difference vector integrator 7 integrates 
the detected difference vectors Sm transferred 
from the image movement detector 5, and pro- 
duces an integrated difference vector signal Smi 

56 which will be transferred to a field memory control- 
ler 9 and an image interpolator 10. 

The tield memory controller 9 produces a sec- 
ond reading address signal Sc based on the signal 
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Smi. The second reading address signal Sc defines 
the address for reading the image signal from the 
field memory 4 to output a video signal Svc having 
the number of pixels required in the final video 
output to the interpolator 10. The interpolator 10 
interpolates the video signal Svc using an interpola- 
tion coefficient determined by the integrated dif- 
ference vector signal Smi and outputs the resul- 
tants as the final video signal SvcV 

It is to be noted that the image movement 
detector 5 may be constructed as described by K, 
Uomori. et al., in "Automatic Image Stabllizlnig Sys- 
tem by Fuil-Digital Signal Processing" (IEEE Trans- 
actions on Consumer Electronics, Vol. 36, No. 3, 
August 1990 (pp. 510 - 519)), and further detailed 
description is therefore omitted below. The detec- 
tion precision of the detected difference vector 
signal Sm in this embodiment is better than one 
pixel horizontally and one line vertically, which will 
be described later with reference to Fig. 6. 

In addition, the image pick-up device controller 
8 may be constructed as described by T. 
Kinugasa, et a!., in "Electronic Image Stabilizer for 
Video Camera Use" (IEEE Transactions on Con- 
sumer Electronics, Vol. 36. No. 3, August 1990 (pp. 
520 - 525)). and further detailed description is 
therefore omitted. 

Referring to Figs. 7A. 7B. and 7C, the images 
la, lb, and Ic obtained In the pick-up device 1 , the 
field memory 4. and the final output Image, respec- 
tively, are illustrated. Fig. 7D shows all images la, 
lb, and Ic as superimposed. As shown In Fig. 7A, 
the picked up image la in the image pick-up device 
1 Is represented by the image matrix having the 
number Ah of pixels horizontally and the number 
Av of lines vertically. Similarly, the images lb and 
Ic are represented by the matrix having the number 
Bh of pixels and the number Bv of vertical lines 
and the matrix having the number Ch of pixels and 
the number Cv of lines, respectively. As shown in 
Fig. 7D, the relationship between the number of 
pixels Ah, Bh, and Ch and the number of lines Av, 
Bv, and Cv Is defined by the following equations. 

Ah>Bh>Ch (1). 

Av>Bv>Cv (2). 

(Ah x Av) > (Bh X Bv) > (Ch x Cv) (3). 

For example, to obtain a final output image Ic 
with a horizontal resolution of approximately 400 
TV lines assuming values of Ch = 670 pixels and 
Cv = 242 lines, Bh and Bv are values larger than 
Ch and Cv, respectively, e.g.. Bh = 720 pixels and 
Bv = 250 lines. Ah and Av are values larger than 
Bh and Bv, respectively, e.g., Ah = 800 pixels and 
Av = 300 lines. Av is the number of vertical lines 



In the image pick-up device 1 . Bv is the number of 
lines in the video signal Svb output therefrom. It Is 
to be noted that Av does not necessarily conform 
to the number of pholodiode cells in the vertical 

5 direction of the pick-up device 1 . 

When first reading the video signal Svb from 
the image pick-up device 1 for the image move- 
ment prediction, the area equivalent to the matrix 
of pixels (Bh X Bv) for representing the image lb in 

10 the field memory 4 is read according to the image 
movement prediction. When next reading the video 
signal Svc from the field memory 4 for^the image 
movement detection, the area equivalent to the 
matrix of pixels (Ch x Cv) required for representing 

76 the image Ic Is read according to the movement of 
the image. As a result, an enlargement process 
such as an electronic zooming method is not re- 
quired to obtain the final video output signal Svc. It 
is to be noted that if Ah, Bh, and Ch, and Av, Bv, 

20 and Cv are respectively equal values such that Ah 
= Bh = Ch and Av = Bv = Cv are satisfied, the 
Image frame will be reduced each time the video 
signals Svb and Svc are read for the image move- 
ment compensation. In this case, an enlargement 

25 process such as an electric zooming will be re- 
quired, resulting in the deterioration of the image. 

Referring to Fig. 8. a simulation used to de- 
scribe reading the video signal from the field mem- 
ory 4 is shown. When there is no image move- 

30 ment, the video signal Svc corresponding to the 
Image Ic indicated by a solid line is read from the 
center part, for example, of the image lb stored to 
the field memory 4. In this case the image Is read 
from the reading start point Pr at the top left corner 

35 of the image Ic. 

When the image Ic moves diagonally right di- 
rection as shown by a vector Mv\ this movement is 
detected. To compensate this movement, the read- 
ing start position Pr is shifted according to the 

40 detected movement vector l^v'. 

Similarly, when the image Ic further currently 
moves as shown by vector Mv, the reading start 
position is further shifted according to the detected 
movement vector Mv. 

45 The movement of image indicated by move- 
ment vector Mv here can be detected as the de- 
tected difference vector signal Sm in this embodi- 
ment, as described in the above. Then, the video 
signal read position is shifted, reference to the 

50 Starting position Pr of the read area, according to 
the integrated value of the detected difference vec- 
tor Smi. The integrated value of the detected dif- 
ference vectors Sm correspond to pixels and line 
units in the read out image. Therefore, when the 

55 integrated value is an integer, the Image having an 
outline expressed by pixels can be read out. How- 
ever, when the integrated value is decimal value, 
the image having an outline fallen between pixels 
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or lines shall be read out. For this purpose, the 
present Invention provides an image Interpolator, 
whose detailed construction will be described later. 

Referring to Fig. 2, the image movement pre- 
dictor 6 of Fig. 1 Is shown. The image movent 
predictor 6 Includes a movement vector calculator 
62 for producing an actual movement vector signal 
Sma representing the actual movement of the Im- 
age based on the detected difference vector signal 
Sm. A movement predictor 61 produces a predic- 
tion movement vector signal Smp based on the 
predicted movement signal Sma. This signal Smp 
is transferred to a prediction vector Integrator 63 
and, at the same time, is returned to the movement 
vector calculator 62. The prediction vector Integra- 
tor 63 produces the predicted movement signal Sp 
by integrating the prediction movement signal 
Smp. The operation of the Image movement pre- 
dictor 6 will be described later with reference to 
Fig. 4. 

Referring to Fig. 3. a modification of the Image 
movement predictor 6 is shown. In this modifica- 
tion, the movement predictor 61 is removed from 
the Image movement predictor 6 of Fig. 2. As a 
result, the actual movement vector signal Sma is 
directly transferred to the prediction vector integra- 
tor 63. and is returned to the movement vector 
calculator 62. The operation of the Image move- 
ment predictor 6 thus formed is described later. 

Referring to Fig. 4, the image movement pre- 
dictor 6 of Fig. 2 is shown. The movement predic- 
tor 61 has a first multiplier 6102 which multiplies 
the actual movement vector signal Sma by two and 
outputs thus multiplied movement vector signal 
2Sma to an adder 6104. The movement predictor 
61 further has a delay circuit 6101 for delaying the 
actual movement vector signal Sma by a delay 
period equal to one field of the video signal, I.e.. 
1/60 second, and for producing a delayed actual 
movement vector signal Sma'. A second multiplier 

6103 is provided for multiplying the delayed actual 
movement vector signal Sma' by -1 and transfer- 
ring the negative and delayed actual movement 
vector signal -Sma' to the adder 6104. The adder 

6104 adds these signals 2Sma and -Sma' and 
produces the predicted motion vector signal Smp. 
Therefore, the movement predicted movement vec- 
tor signal Smp can be expressed by the equation 
of 

Smp = 2Sma - Sma' (4). 

This predicted movement vector signal Smp is 
output to the movement vector calculator 62 and 
the prediction vector integrator 63, as described 
previously. 

The movement vector calculator 62 has a delay 
circuit 6201 which delays the predicted movement 



vector signal Smp for a delay period equal to one 
field of the video signal, i.e., 1/60 second, to pro- 
duce a delayed predicted movement vector signal 
Smp'. An adder 6202 adds this delayed predicted 
5 movement vector signal Smp' to the detected dif- 
ference vector signal Sm coming from the image 
movement detector 5. The resulting actual move- 
ment vector Sma is output to the movement pre- 
dictor 61 which uses this vector signal Sma to 
10 generate and output the predicted movement vec- 
tor signal Smp in a feedback loop to the movement 
vector calculator 62, as show in Fig. 4. 

The prediction vector integrator 63 has a delay 
circuit 6301 for delaying the signal Input thereto for 
15 a delay period equal to one field of the video 
signal, i.e., 1/60 second. An adder 6302 is provided 
for adding the signal output from the delay circuit 
6301 to the predicted movement vector signal Smp 
and produces an integrated predicted movement 
20 vector signal Sp. The delay circuit 6301 produces 
the delayed integrated predicted movement vector 
signal Sp'. The adder 6302 and delay circuit 6301 
thus form a feedback loop whereby the predicted 
movement vectors signal Smp are integrated each 
25 field to generate the integrated predicted move- 
ment vector signal Sp. It Is to be noted that this 
integrated signal Sp is also the movement predic- 
tion signal and is transferred to the image pick-up 
device controller 8, as described previously. 
30 Referring to Fig. 5, an alternation of the move- 
ment predictor 61 is shown. This alternative move- 
ment predictor 61 has two delay circuits 6101 and 
6106 for delaying the actual movement vectors by 
a delay period is equal to one field of the video 
35 signal, i.e., 1/60 second. The first delay circuit 6101 
delays the actual movement vector signal Sma 
from the movement vector calculator 62 and pro- 
duces the delayed actual movement vector signal 
Sma'. The second delay circuit 6106 further delays 
40 this delayed signal Sma* and produces the twice 
delayed actual movement vector signal Sma". 

Three multipliers 6107, 6108, and 6109; and 
two adders 6104 and 6111 are further provided. 
The first multiplier 6107 multiplies the actual move- 
45 ment vector Sma from the movement vector 62 by 
3 and outputs this tripled signal 3Sma to the first 
adder 6104. The second multiplier 6108 multiplies 
the delayed actual movement vector signal Sma' 
by - 3 and outputs this negative-tripled signal 
50 -3Sma' to the first adder 6104. The third multiplier 
6109 multiplies the twice delayed actual movement 
vector signal Sma" by 1 and outputs the resulted 
signal Sma" to the second adder 6111. 

The first adder 6104 adds the signals from the 
65 first and second multipliers 6107 and 6108, and 
outputs to added signal expressed as 3(Sma - 
Sma') to the second adder 6111. The second ad- 
der 61 1 1 adds the signal Sma" to the added signal 
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3(Sma - Sma*) and produces the predicted move- 
ment signal Smp, Therefore, the movement pre- 
dicted movement vector signal Smp, according to 
the alternative movennent predictor 61 , can be ex- 
pressed by the equation of 

Smp = 3(Sma - Sma") + Sma" (5). 

The operation of thus constructed movement pre- 
dictor 61 will be described later. 

Referring to Fig. 6, the image interpolator 10 of 
Fig. 1 is shown. In this Figure, only the part for 
interpolating in the vertical direction is shown, for 
the sake of brevity. The innage interpolator 10 has 
an output selector 1004; three one-line memories 
1001, 1002, and 1003; a signal selector 1005; a 
control circuit 1006; two multipliers 1007 and 1008; 
an adder 1009; and a coefficient gienerator 1010. 

The output selector 1004 has two input ports 
each connected to the field menriory 4 for receiving 
the video signal Svc therefrom and to the control 
circuit 1006 for receiving a control signal therefrom, 
respectively. The output selector 1004 further has 
three output ports OUTPUT 1, OUTPUT 2, and 
OUTPUT 3 connected to an input port of one-line 
memories 1001. 1002. and 1003. Each of the one- 
line, memory devices 1001, 1002, and 1003 is 
capable of storing one horizontal scan line of the 
video signal Svc. At any given time, one of the 
three one-line memories 1001, 1002, and 1003 is 
used for writing, and two are used for reading. The 
output selector 1004 sequentially outputs the video 
signal Svc, which is output one line at a time from 
the field memory 4, one line at a time to the one- 
line memories 1001. 1002, and 1003. 

The signal selector 1005 has three input ports 
INPUT 1, INPUT 2, and INPUT 3 connected to an 
output port of one-line memories 1001, 1002, and 
1003. The control circuit 1006 has five output ports 
each connected to the output selector 1004, one- 
line memories 1001, 1002, and 1003, and the sig- 
nal sector 1005. Thus, the control circuit 1006 
controls the input/output selections of the output 
selector 1004 and signal selector 1005, as well as 
the read/write operation of one-line memories de- 
vices 1001, 1002, and 1003. 

The signal selector 1 005 further has two output 
ports OUTPUT V and OUTPUT 2' each connected 
to the first multiplier 1007 and the second multiplier 
1008, respectively. The signal selector 1005 out- 
puts the video signals Svc read from the two one- 
line memory devices not currently being written to 
through OUTPUT V and OUTPUT 2\ Of the two 
lines of the video signal Svc Input to the signal 
selector 1005, the signal that is output first from the 
field memory 4 is output through OUTPUT 2'. 

The coefficient generator 1010 has one input 
port for receiving the integrated difference signal 



SmI from the difference vector integrator 7 and 
produces interpolation coefficient signals W and 1 - 
W bases on the signal SmI. The coefficient gener- 
ator further has two output ports each connected to 

5 the multipliers 1007 and 1008 for transferring the 
coefficient signals W and 1-W, respectively. The 
third adder 1009 is connected to both the first and 
second multipliers 1007 and 1008 for adding the 
signals output therefrom to produce the final video 

10 signal Svc. 

By thus using one of the three one-line mem- 
ory devices for writing and two for reading, and 
sequentially switching which one-line memory de- 
vices are used for the current read and write oper- 

16 ations, it is possible to achieve real-time signal 
processing. 

Multipliers 1007 and 1008 multiply the two 
video signal lines read from two of the three one- 
line memories 1001, and 1002, and 1003 by inter- 

20 polation coefficients W and 1 - w, respectively. 
These coefficients are generated by the coefficient 
generator 1010 based on the output from the de- 
tected difference vector integrator 7. 

The adder 1009 then adds the output signals of 

25 the multipliers 1007 and 1008 to obtain the inter- 
polation result, which is thus output from the image 
interpolator 10 as the final video signal Svc'. 

The operation of the image movement com- 
pensation apparatus MC thus comprised is de- 

30 scribed below. 

The image movement detector 5 performs a 
correlation operation on the video signal of the 
current field obtained by the image pick-up device 
1 and the video signal from the previous field, and 

35 detects any movement (the detected difference 
vector signal Sm) between the image fields based 
on the result of this correlation operation. This 
detected difference vector signal Sm is the dif- 
ference between the image movement (predicted 

40 movement signal Smp) predicted by the image 
movement predictor 6 in the previous field and the 
actual movement (actual movement vector signal 
Sma) of the image in the current field. Furthermore, 
if the movement (predicted movement signal Smp) 

45 predicted by the image movement predictor 6 and 
the actual movement (actual movement vector sig- 
nal Sma) are equal, the movement of the image 
detected by the image movement detector 5 (de- 
tected difference vector signal Sm) is zero (0). 

50 As a result, the image movement detector 5 
actually detects the error in the amount of move- 
ment predicted between the previous field and the 
current field rather than the actual movement be- 
tween the Images of the previous and current 

55 fields. Predicting the movement between the cur- 
rent and next fields is therefore based on the actual 
movement vector between the previous and current 
fields, and is calculated from the detected dif- 
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ference vector obtained by the Image movement 
detector 5. 

This actual movement vector Sma is obtained 
by the movement vector calculator 62 (Fig. 4) such 
that the predicted movement vector Smp* obtained 
in the previous field is added to the detected 
difference vector Sm obtained by the image move- 
ment detector 5 is added, as described previously 
with reference to Fig. 4 . The movement predictor 

61 predicts the movement vector (predicted move- 
ment vector signal Smp) between the current and 
next fields from the actual movement vector (Sma) 
obtained by the movement vector calculator 62. 
and outputs this movement vector as the predicted 
movement vector (predicted movement vector sig- 
nal Smp) to the prediction vector integrator 63, 

When the image movement predictor 6 is con- 
figured as shown in Fig. 3, where no movement 
predictor 61 of Fig. 2 are provide. The actual move- 
ment vector Sma between the previous and current 
fields obtained by the movement vector calculator 

62 is used as the predicted movement vector in- 
stead of signal Smp of Fig. 2. it is possible to filter 
out the low frequency component of the detected 
image movement. Then the prediction vector in- 
tegrator 63 integrates the actual movement vector 
Sma. Thus configured image movement predictor 6 
is particularly effective for the movement prediction 
of such an image of object moving with a constant 
velocity. 

When the image movement predictor 6 is con- 
figured as. shown in Fig. 4. the movement predictor 
61 performs the following operation. The input sig- 
nal is multiplied twice by the first multiplier 6102 
(signal 2Sma). The Input signal Sma is delayed by 
the delay circuit 6101 (signal Sma') and multiplied 
by -1 by the second multiplier 6103 (signal -Sma'). 
These signals 2Sma and -Sma' are added by the 
adder 6104, and the summed signal 2Sma - Sma' 
is output as the predicted movement vector Smp. 
Thus configured image movement predictor 6 is 
useful for the movement prediction of such an 
image of object moving with a constant accelera- 
tion. 

The movement prediction by thus configured 
movement predictor 61 of in Fig. 4 is further de- 
scribed below with reference to Figs. 9A, 9B, and 
9C. Note that the vectors used in Figs. 9A, 9B, and 
9C are expressed as one dimensional values. The 
actual movement vector Sma of each field input to 
the movement predictor 61 is expressed with (n) 
where 'n' Is the field number. When the current 
field is field (n), the actual movement vector Sma 
of the current field has the value of 2. The delayed 
movement vector Sma' which is the vector Sma of 
the previous field has the value of 1. for example. 
The predicted movement vector Smp output from 
the movement predictor 61 in this filed will be 3. 



This is basically extrapolation of the input by 
means of a primary function based on the assump- 
tion that the change in the movement vector can 
be approximated using a primary function. 

6 When the image movement predictor 6 has the 

movement predictor 61 which is configured as 
shown in Fig. 5, the input signal multiplied by 3 by 
the first multiplier 6107 is added by the first adder 
6104 with the input signal Sma delayed by the 

10 delay circuit 6101 and multiplied by -3 by the 
second multiplier 6108. This sum signal is then 
added by the second adder 6111 with the Input 
signal delayed by both delay circuits 6101 and 
6106 and multiplied by 1 by the third multiplier 

IS 6109. and the sum signal from the second adder 
61 1 1 is output as the predicted movement vector. 
Thus configured image movement predictor 6 is 
useful for the movement prediction of such an 
image of object moving with a variable accelera- 

20 tion. 

The movement prediction by thus configured 
movement predictor 61 is further described below 
with reference to Figs. 10A, 10B. and IOC. Note 
that the vectors used in Fig. 10 are expressed in a 

25 manner similar to that of Figs. 9A, 98, and 9C. 
When the current field is field (n), the actual move- 
ment vector Sma of the current field has the value 
of 4, the delayed movement vector Sma'(n-I) 
which is the vector Sma of the previous field is 2. 

30 And, the twice delayed movement vector Sma" (n- 
2) which is the vector Sma of the field two fields 
before is 1, The predicted movement vector Sma 
output from the movement predictor 61 will be 7. 
This is basically extrapolation of the input by 

35 means of a secondary function based on the as- 
sumption that the change in the movement vector 
can be approximated using a secondary function. 

By using the image movement predictor 6 of 
either the configuration shown in Fig. 4 or Fig. 5, 

40 the high frequency component of image movement 
can be more efficiently removed than when the 
actual movement vector Sma between the previous 
and current fields is used as the predicted move- 
ment vector Smp by the image movement predic- 

45 tor 6 shown in Fig. 3. It is needless to say that the 
image movement predictor 6 having the movement 
predictor 61 of Fig. 5 can be used tor predicting 
the movement of an object moving at a constant 
velocity. 

50 Using the predicted movement vector Smp 
thus obtained, the prediction vector integrator 63 
calculates the integrated value (integrated predict- 
ed movement vector Sp) of the predicted move- 
ment vectors Smp obtained in each field. This 

55 integrated predicted movement vector Sp is output 
to the image pick-up device controller 8, which 
controls the video signal read address of the Image 
pick-up device 1 by the first reading address signal 
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Sb produced based on the integrated predicted 
movenrient vector Sp so as to compensate for 
image movement. 

The detected difference vectors Sm obtained 
by the image movement detector 5 are also in- 
tegrated by the difference vector integrator 7, and 
the integrated difference signal Smi is output there- 
from. The difference vector integrator 7 can be 
constructed Identically to the prediction vector in- 
tegrator 63 shown in Fig. 4, and is therefore not 
specifically shown in the figures. The field memory 
controller 9 produces the second reading address 
signall' Sc based on the integrated difference signal 
Smi. Therefore, by controlling the video signal read 
address of the field memory 4 with the second 
reading address signal Sc. any residual movement 
component left by movement prediction in the vid- 
eo signal Svb read from the image pick-up device 
1 can be removed. 

Note that this can be simply accomplished 
when the output (signal Smi) from the difference 
vector integrator 7 is an integer value, i.e., the 
value of a specific pixel or line. If the output (signal 
Smi) from the difference vector integrator 7 in- 
cludes a decimal value, which is equivalent to a 
value indicating a position between pixels or be- 
tween lines, the decimal part must be rounded 
because reading from the field memory 4 is only 
possible in pixel or line units. The precision of the 
movement compensation operation, and therefore 
the image quality, thus deteriorates. This is avoided 
by the interpolation process applied by the image 
interpolator 10 to maintain cornpensation precision 
better than one pixel or one line. 

With reference to Figs. 6 and 11, he interpola- 
tion process of the image interpolator 10 is de- 
scribed for interpolation in the vertical direction. 

Interpolation is not required when an integer 
value (signal Smi) is output from the difference 
vector integrator 7 in a given field. This is because 
an integer value indicates a line position, and it is 
therefore possible to simply read the video signal 
from the line determined by the signal Smj output 
from the difference vector integrator 7. 

However, when this output value contains a 
decimal part, it is necessary to start reading from a 
position z between lines k and (k + 1) of the video 
signal Svb stored to the field memory 4 as shown 
in Rg. 11. But because reading from the field 
memory 4 is only possible in line units, it is not 
possible to read from position z. Therefore, the 
video signal Svc is read from the field memory 4 is 
read from the position z, as shown in Fig. 11. 
Without the image Interpolator 10, it is necessary in 
this case to round the decimal part, begin reading 
from line k or (k + 1), and use the read image Svc 
as the final output image Svc'. In this case com- 
pensation precision will deteriorate by an amount 



equivalent to the decimal part that was rounded. 

This problem can be eliminated, however, by 
the image interpolator 10 shown in Rg. 6 syn- 
thesizing the line at position z by interpolation as 
5 described bellow. It is assumed below that position 
z is between lines k and (k + 1 ) of a given field in 
the video signal stored in the field memory 4 (the 
distance between lines being 1). The ratio of the 
distance from position z to line k and from position 

10 z to line (k + 1) is w:(1 - w), where 0 ^ w ^ 1, as 
shown in Fig. 11. To interpolate the image line at 
this position z, the field memory controller 9 sets 
the line from which reading from the field memory 
4 starts based on the output from the difference 

75 vector integrator 7 (line k in this example), and 
reading begins. 

The video signals Svc read from line k are 
sequentially input to the image interpolator 10 and 
stored in one of line nnemories 1001, 1002, and 

20 1003. It is here assumed that line k is stored to line 
memory 1003 and line (k + 1) is stored to line 
memory 1002. As described above, the signal se- 
lector 1005 thus outputs the line k signal which was 
read first from OUTPUT 2, and the line (k + 1) 

25 signal from OUTPUT 1. The interpolation coeffi- 
cients are generated by the coefficient generator 
1010 based on the signal Smi, and are input to the 
multipliers 1007 and 1008. The line k signal is thus 
multiplied by interpolation coefficient (1 - w), where 

30 w is the decimal part of the output of the detected 
difference vector integrator 7. The line (k + 1) 
signal is multiplied by interpolation coefficient w. 
and the results are added by the adder 1009 to 
obtain the interpolated video signal Svc'. 

35 The interpolated video signal Svc' is thus ob- 

tained by reading lines k and (k + 1) from two 
one-line memory devices, multiplying each by pa- 
rameters (w and (1 - w) in this case) determined by 
a coefficient according to the distance of the inter- 

40 polated line from the read lines, and adding the 
resulting products, as shown in Fig. 6. 

It is therefore possible to achieve high preci- 
sion movement compensation by means of an im- 
age movement compensation apparatus MO, as 

45 described above when compensating for image 
movement introduced to a recorded image by cam- 
era movement. This is specifically possible by pro- 
viding the image pick-up device 1 having more 
pixels than a conventional pick-up device of which 

50 the number of pixels is determined according to 
the broadcast system or the system operating 
clock; the field memory 4 with a pixel storage 
capacity greater than the number of pixels required 
for the final video output; the image movement 

55 predictor 6 for predicting the movement from the 
current field to the next image; and the image 
interpolator 10 for interpolating the video signal. 
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With this configuration, image movement is 
compensated for by the image movement predictor 
6 predicting the amount of Image movement in the 
next field and shifting the video signal read address 
of the image pick-up device 1. The video signal 
Svb is then read from the field memory 4 based on 
the Integrated value Sp of the detected difference 
vectors Sm obtained by the Image movement de- 
tector 5, and interpolated by the Image interpolator 
10 to achieve movement compensation precision 
better than one pixel or one line. The detected 
difference vector Sm is equivalent to the residual 
image movement that could not be completely 
removed by predictive offsetting of the image pick- 
up device read address. This process eliminates 
the need for image enlargement (electronic zoom- 
ing) of the video signal after movement compensa- 
tion as required by conventional processes, and 
thus also prevents the Image deterioration accom- 
panying this image enlargement (electronic zoom- 
ing) process. 

It will also be obvious that the above circuit 
functions can be achieved in software alone. 

In addition, the image movement detector 5 
and Image pick-up device drive controller 8 are as 
in the prior art, but shall not be so limited. 

The movement predictor 61 Is also described 
as being one of three different configurations 
shown in Figs. 3. 4, and 5, but shall not be so 
limited and can use a method of appropriately 
selecting the output of any of these three configu- 
rations. In addition, while two adders are used in 
the second configuration of the movement predic- 
tor 61 of Fig. 5, it will be obvious that the same 
configuration can be achieved using only one ad- 
der. 

Image movement detection and prediction are 
also executed on a per field basis in the above 
embodiment, but the Invention shall not be so 
limited. These operations can. for example, also be 
applied each frame instead of each field. 

The number of vertical lines of the pick-up 
device 1 in Figs, 7A, 7B, 7C. and 7D is defined as 
Av above, but this is specifically the number of 
lines In the video signal obtained from the image 
pick-up device, and will obviously not necessarily 
match the number of photodiode cells on the verti- 
cal axis of the image pick-up device. 

Furthermore, while the interpolation process in 
the horizontal direction is not specifically described 
in the above embodiment, an identical interpolation 
process can be achieved by substituting one pixel 
memory devices for the one line memory devices 
in Fig. 6, and controlling video signal read/write 
operations on a pixel unit basis. Just as one exam- 
ple for this purpose, one line memories in Fig. 6 
are replaced by one pixel memories. 



Referring to Fig. 12, an image movement com- 
pensation apparatus according to a second em- 
bodiment of the present invention is shown. The 
differences between the apparatus MC of Fig. 1 
5 and the apparatus MC of Fig 12 are described 
below. 

When compared with the apparatus MC, the 
apparatus MC is further provided with a first at- 
tenuation compensator circuit 11, as shown in Fig. 
10 12. Furthermore, the image movement predictor 6 
of the apparatus MC is replaced by an image 
movement predictor 12 additionally provided with a 
second attenuation compensator 1204 and an at- 
tenuator 1205 in the apparatus MC, as shown in 

75 Fig. 13. 

As a result, the signal Smi supplied from the 
difference vector integrator 7 to the field memory 
controller 9 and the Image interpolator 10 In the 
apparatus MC is changed to go through the attenu- 

20 ation compensator 11. But, the operations of the 
field memory controller 9 and the image interpola- 
tor 10 remain as described in the first embodiment. 

Furthermore, the signal Sma supplied from the 
movement vector calculator 62 to the movement 

25 predictor 61 in the first embodiment (Fig. 2 is 
changed to go through the second attenuation 
compensator 1204. Note that the movement vector 
calculator 62 and the movement predictor 61 op- 
erate the same in the first embodiment, respec- 

30 tively. And the signal Sp supplied form the predic- 
tion vector Integrator 63 (the image movement pre- 
dictor 6) to the image pick-up device controller 8 is 
changed to go though the attenuator 1205. 

Further description of components having the 

35 same operation and effect in the second embodi- 
ment as in the first embodiment is omitted below. 
Only the differences from the first embodiment are 
described. 

As shown in Fig. 13. the attenuator 1205 has 
40 an multiplier 12052 and an adder 12051 connected 
to each other. The signal Sp from the prediction 
vector integrator 63 is input to the multiplier 12052 
and the adder 12051. The multiplier 12052 mul- 
tiplies the signal Sp with an attenuation coefficient 
45 a, where 0 ^ a < 1, and outputs the resultant aSp 
to the image pick-up device controller 8 and the 
adder 12051. The adder 12051 reduce this resul- 
tant signal aSp from the signal Sp and produces an 
attenuation signal Spa representing the attenuation 
50 level of this process (Spa = Sp(l-a)). This attenu- 
ation signal Spa is transferred to the first attenu- 
ation compensator 11 and the second compensator 
1204. 

The second attenuation compensator 1204 has 
55 the adder 12041 which adds the input Sma from 
the movement vector calculator 62 and the inverse 
of the output Spa from the attenuator 1205, and 
outputs the result Smc to the movement predictor 
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61. 

This attenuation process is used to assure suf- 
ficient suppression of image movennent and move- 
ment compensation over a sufficiently broad area 
when Image movement compensation Is applied. 
This process is described in detail In "Fluctuation 
stabilizing apparatus," United States patent number 
5099323 issued March 24, 1993 to Morlmura et al. 
Therefore, further description is omitted. As shown 
iri Fig. 14, the first attenuation compensator 11 has 
an adder 1101 for attenuating the signal SmI from 
the difference vector integrator 7 by an amount 
determined by and the signal Spa from the image 
movement predictor 12. The first attenuation com- 
pensator circuit 1 1 outputs the signal Sa having an 
attenuated integrated value to the field memory 
controller 9 and to the image interpolator 10. 

In operation, the attenuator 1205 attenuates the 
integrated value (integrated predicted movement 
vectors Sp) integrated by the prediction vector 
integrator 63, i.e., effectively reduces the move- 
ment compensation. As a result, the Image move- 
ment component left in the video signal Svb read 
from the Image pick-up device 1 increases. It must 
be noted, however, that this residual Image move- 
ment component in the video signal Svb from the 
image pick-up device 1 is detected by the image 
movement detector 5. Furthermore, this residual 
image movement component is removed by the 
video signal read operations from the field memory 
4 and the interpolation process of the image inter- 
polator 10. As a result, the effect of deliberately 
reducing movement compensation by this attenu- 
ation process does not appear in the final output 
image (signal Svc'). Therefore, this deliberate re- 
duction in compensation (attenuation a of the in- 
tegrated predicted movement vector Sp) by the 
attenuator 1205 when the video signal Svb Is read 
from the image pick-up device 1 is removed from 
the output Smi of the difference vector integrator 7 
by the first attenuation compensator 11. 

In addition, because uncompensated move- 
ment components resulting from this deliberate at- 
tenuation process are contained In the detected 
difference vector Sm detected by the image move- 
ment detector 5 in addition to the normal error of 
the movement prediction process, there is a dif- 
ference between the Inter-fletd image movement 
calculated by the movement vector calculator 61 
and the actual image movement, and the precision 
of the predicted movement is impaired. 

By means of the present embodiments de- 
scribed above, attenuation of the Integrated pre- 
dicted .movement vector Sp, which is used to suffi- 
ciently suppress Image movement and enable 
movement compensation over a broad area, can be 
used during image movement compensation with- 
out impairing the effectiveness of the attenuation or 



the precision of image movement compensation. 

It is to be noted that the present embodiment 
has been described using the first attenuation com- 
pensator 11 and second attenuation compensator 

5 1204 as discrete components, but the Invention 
shall not be so limited and it is possible to combine 
these Into a single attenuation compensator circuit. 

Furthermore, the attenuation coefficient a is 
described as a constant value above, but shall not ^ 

10 be so limited and may, for example, be variable 
according to the magnitude of the integrated pre- 
dicted movement vector Sp. 

It will also be obvious that the above functions 
can be achieved in software alone. 

75 Although the present invention has been fully 
described in connection with the preferred embodi- 
ments thereof with reference to the accompanying 
drawings, it is to be noted that various changes and 
modifications are apparent to those skilled in the 

20 art. Such changes and modifications are to be 
understood as Included within the scope of the 
present invention as defined by the appended 
claims unless they depart therefrom. 

25 Claims 

1. An image movement compensation apparatus 
(MC) for compensating the movement of an 
image comprising: 

30 an image producing means (1 ) for produc- 

ing a first image (la) representing an object 
expressed by a first predetermined number 
(Ah X Av) of pixels and reading out a second 
image (lb) expressed by a second predeter- 

35 mined number (Bh x Bv) smaller than said first 
predetermined number (Ah x Av) of pixels 
therefrom; 

a memory means (4) for storing said sec- 
ond image (lb) and reading out a third image 
40 (Ic) expressed by a third predetermined num- 

ber (Ch X Cv) smaller than said second pre- 
determined number (Bh x Bv) of pixels there- 
from: 

a movement signal means (5) for produc- 
es ing a vector signal (Sm) representing the 
movement of said Image at intervals of pre- 
determined period (field or frame); 

a prediction signal means (7 and 6) for 
producing first and second prediction signals 
50 (Smi and Sp) predicting the movements which 

said first and second images (la and lb) which 
will make In said predetermined period, re- 
spectively, based on said vector signal (Sm); 
and 

56 a designation means (9 and 8) for des- 

ignating areas of said memory means (4) and 
said image producing means (1) from which 
said third and second images (Ic and lb) are 
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read out based on said first and second pre- 
diction signals (Smi and Sp), respectively. 

2. An image movement compensation apparatus 

(MC) for compensating the movement of an 5 4. 
image representing an object, said apparatus 
comprising: 

an video signal producing means (1) for 
producing a first image (la) of said object ex- 
pressed by a first predetermined number (Ah x io 
Av) of pixels and reading out a second image 
(lb) from said first Image (la) to produce a first 
video signal (Svb), said second image (lb) be- 
ing expressed by a second predetermined 
number (Bh x Bv) smaller than said first pre- is 5. 
determined number (Ah x Av) of pixels; 

a memory means (4) for storing said first 
video signal (Svb); 

an image movement detecting means (5) 6. 
for detecting the movement of said second 20 
image (lb) at intervals of a predetermined pe- 
riod (field or frame) and producing an move- 
ment vector signal (Sm); 7. 

an Integrating means (7) for integrating 
said movement vector signal (Sm) and produc- 25 
ing an integrated movement vector signal 
(Smi); 

a first control means (9) for controlling said 
memory means (4) to read out a third image 
(Ic) from said second image (lb) based on said 30 
integrated movehnent vector signal (Smi) and 
producing a second video signal (Svc), said 
third image (Ic) being expressed by a third 
predetermined number (Ch x Cv) smaller than 
said second predetermined number (Bh x Bv) 35 
of pixels; ® 

an image movement prediction means (6) 
for predicting the movement which said sec- 
ond image (lb) at intervals of said predeter- 
mined period (field or frame) based on said 40 
movement vector signal (Sm) and producing a 
movement prediction signal (Sp); and 

a second control means (8) for controlling 
said image signal producing means (1) to 
change the address from which said second 45 
image (lb) are read out based on said move- 
ment prediction signal (Sp). 

3. An image movement compensation apparatus 
(MC) as claimed in Claim 2, wherein said im- so 
age movement prediction means (6) com- 
prises: 

a movement vector prediction signal 
means (62 and 61) for producing a prediction 
vector signal (Smp) predicting a movement of 55 
said first image (la) after one of said predeter- 
mined period (field or frame); and 

a prediction vector integrating means (63) 



for Integrating said prediction vector signal 
(Smp) to produce said movement prediction 
signal (Sp). 

An image movement compensation apparatus 
(MC) as claimed in Claim 3, wherein said 
movement vector prediction signal means (62 
and 61) produces said prediction vector signal 
(Smp), based on said movement vector (Sm) 
and a delayed prediction vector signal (Smp') 
which is the same as said prediction vector 
signal (Smp) before a single of said predeter- 
mined period (field or frame). 

An image movement compensation apparatus 
(MC) as claimed In Claim 4, wherein said pre- 
determined period is a field. 

An Image movement compensation apparatus 
(MC) as claimed in Claim 4, wherein said pre- 
determined period is a frame. 

Ari image movement compensation apparatus 
(MC) as claimed in Claim 2. further comprising 
an Interpolating means (10) for interpolating 
said second video signal (Svc) based on said 
integrated movement vector signal (Smi) and 
producing a third video signal (Svc*) represent- 
ing a fourth image, whereby when said in- 
tegrated movement signal (Sm) has a decimal 
value, the outline of said fourth image fallen 
between pixels are expressed by pixels inter- 
polated according to the pixels opposite sides 
with respect to said outline. 

An image movement compensation apparatus 
(MC) as claimed in Claim 2, further compris- 
ing: 

a attenuating means (1205) for attenuating 
said movement prediction signal (Sp) to pro- 
duce an attenuation movement prediction sig- 
nal (Spa); and 

a reducing means (11) for reducing said 
integrated vector signal (Smi) by said attenu- 
ation movement signal (Spa) to produce an 
attenuated vector signal (Sa). 
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@ Image movement compensation apparatus. 



© An image movement compensation apparatus 
(MC) maintaining high image quality is achieved 
using a filed memory (4) having more pixels than the 
number of final output image (Svc) which is deter- 
mined according to the broad cast system or the 
system operation clock. The image movement com- 
pensation apparatus (MS) further has an Image pick- 
up device (1) having more pixels than the filed 
memory (4), an image movement detector (5). and 



an image movement predictor (6). The image (la) 
obtained by the image pick-up device (1) is cropped 
according to the image movement predicted by the 
image movement predictor (6). The residual move- 
ment component left in the image (Svb) read from 
the image pick-up device (1) is then detected by the 
image movement detector (5). and is next removed 
by image movement compensation using the field 
memory (4). 
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